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ABSTRACT 

A description is given of the Classroom Organization 
and Management Program, designed to help teachers in grades 1-9 and 
staff developers improve their overall instructional and behavioral 
management skills through planning, implementing, and maintaining 
effective classroom practices. It also seeks to improve student task, 
engagement and reduction of inappropriate and disruptive behavior • 
through well-planned and appropriate academic tasks and activities. 
The program addresses underlying needs of both beginning and 
experienced teachers for more professional development and inservice 
training for teachers who wish to improve their management skills and 
for trainers who wish to provide professional development activities 
for teachers. The format is a series of 3-day teacher workshops 
and/or 7-day trainer workshops that focus on: (1) planning and 
implementing effective strategies for room arrangements; (2) rules 
and procedures; (3) student accountability; (4) consequences and 
incentives; (5) behavior management; and (6) conducting class 
lessons. Participants are taught observation techniques for follow-up 
and feedback. Appended are some questions on the program posed by the 
Program Effectiveness Panel. (JD) 
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SUBMISSION TO THE 
PROGRAM EFFECTIVENESS PANEL 



ABSTRACT 



Classroom nrgani^tinn * n d Management Program 

Goals: The primary goal of the program is to help teachers improve their overall instructional and 
behavioral management skills through planning, implementing, and maintaining effective classroom 
practices. An additional goal is the improvement of student task engagement and reduction of 
inappropriate and disruptive behavior through well-planned and appropriate academic tasks and activities. 

Purposes and needs addressed This program addresses underlying needs of both beginning and experienced 
teachers for more professional development and inservice training in classroom and behavior management. 
It provides materials and inservice training for both teachers who wish to improve their management s kills 
and for trainers who wish to provide professional development activities for teachers. Because many 
teachers, especially beginning teachers, regularly cite classroom management as an ever-present concern 
(c.f., Veenman, 1984), the program addresses an important need for schools, faculties, and students. 

Method of operation; Based on research findings from several experimental studies, the format is a series 
of 3-day teacher workshops and/or 7-day trainer workshops that focus on six areas: planning and 
implementing effective strategies for room arrangements, rules and procedures, student accountability, 
consequences and incentives, behavior management, and conducting class lessons. Participants are taught 
observation techniques for follow-up and feedback. 

Audience: Regular classroom teachers in grades 1-9 are the primary audience for the program. Also, 
administrators, regional educational labs, state departments of education, rad school staff developers 
wishing to design and deliver professional development workshops for teachers in these grades arc an 
intended audience. 

Claims: Claim 1 - Academy? Achievement « rhanges j n ftnowlcdyc and Skill* . 

Students in English and math classes in grades 7 • 9 who were in the classes of 
teachers trained in the classroom management workshops made significantly higher gains 
on achievement tests than students in control group classes. 

Claim 2 = Improvements in Teachers' Attitudes and Behaviors 
Teachers who participated in the classroom management training workshops used the 
effective practices in their classrooms to a greater extent than teachers in the 
control groups. 

Claim 3 - Improvements In Smdcnts' Attitudes and Behaviors 
Students in classrooms of teachers who participated in the workshops had significantly 
less off-task, less inappropriate and disruptive behavior, and greater success in 
lessons than students in the control group classes. 
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SUBMISSION TO THE 
PROGRAM EFFECTIVENESS PANEL 



Program Area: Inservice Training (Classroom management) 
I. BASIC INFORMATION ~ 

A. Program Title. Location: Classroom Organization & Management Program (COMP) 

Box 3*0 Peabody College, Vanderbilt University 
Nashville, TN 37203 
Contact person: Dr. Carolyn M. Evertson, (615) 322-8100 

B. Original Developer: Dr. Carolyn M. Evertson, Peabody College, Vanderbilt University, directed the 
original research program (1977-1981) at the University of Texas, Research & Development Center for 
Teacher Education. Project staff included Dr. Edmund Emmer, Dr. Julie Sanford, Barbara S. Clements, 
road Dr. Murray Worsham. One experimental study was directed by Dr. Emmer (1981-82), and 
Dr. Evertson directed work on the program through the Arkansas Dept. of General Education 
(1982-1985). Dr. Evertson supervised the dissemination, training, and consulted with the school 
districts in adoption of the program in two model sites (1987-1988). 

C. Applicant agency Peabody College, Vanderbilt University. Peabody College is the College of 
Education and Human Development of Vanderbilt University which is a privately funded, non-profit 
institution of higher education. 

D. Years of Projects: 



Years 



Type of studv/activity 



1977- 78 

1978- 79 

1980- 81 

1981- 82 

1982- 85 

1982- 85 

1983- 88 



Descriptive/correlational study • elementary grade (COS) 
Descriptive/correlational study - secondary grades(JHCOS) 
Experimental field study • elementary level (CMIS) 
Experimental field study • secondary level (JMIS) 
Experimental/evaluation study • secondary grades (CMTST) 
Experimental/evaluation studies • elementary grades (CMTET) 
Dissemination - statewide (Arkansas) 

Dissemination - two sites (Greeley, CO & Kentville (Kings Co.) Nova Scotia 



1987-present 



E. Sources and Levels of Funding; 



Period 



Funds 



1983-85 

1987-present 

1987-present 



1977- 78 

1978- 79 

1979- 80 

1980- 81 
1982-83 



$150,000 
200,000 
210,000 
225,000 
8^0C 
30,000 
75329 
20,000 
40,000 



Kentville (Kings Co.) Nova Scotia 



National Institute of Education 
Greeley (Weld Co.) CO 



Arkansas State Department of Education 
In kind contributions from local school districts (Arkansas) 



National Institute of Education 
National Institute of Education 
National Institute of Education 
National Institute of Education 



II. DESCRIPTION OF PROGRAM 



A. Program Goals? 

The primary goal of the program is to help teachers improve their overall instructional and 
behavioral management skills through planning, implementing, and maintaining effective class room 
practices. A related goal is the improvement of student task engagement and reduction of 
inappropriate and disruptive behavior through well-planned and appropriate academic tasks and 
activities. # 

B. Purposes and Needs Addressed: 

This program addresses underlying needs of both beginning and experienced teachers for more 
inservice training in classroom and behavior management. Teachers, especially beginning teachers, 
regularly dte classroom management as an ever-present concern. Not only is there little argument as 
to the importance of good management from a common sense point of view, but research has also shown 
that many management variables are correlated with pupil achievement (Brophy & Good, 1986). Studies 
in the elementary grades (Anderson, Evertson, & Brophy, 1979; Brophy & Evertson, 1976) and in the 
secondary grades (Evertson, Anderson, Anderson, & Brophy, 1980; Evertson & Emmer, 1982) show that the 
more academically effective teachers in those studies have better organized classrooms and fewer 
behavior problems. Additionally, research indicates that the key to managing classrooms effectively 
begins from the first day of school with a systematic approach of advanced preparation and planning 
(Brophy, 1982; Evertson, 1987). This program addresses an important need for schools, faculties, and 
students by focusing on helping teachers plan effective management strategies for starting the year, 
as well as for implementing and maintaining these as the year progresses. 

C. Intended Audience: 

Regular classroom teachers in grades 1-9 are the primary audience for the program. Also, 
administrators, instructional supervisors, regional educational labs, and state departments of 
education wishing to design and deliver professional development workshops for teachers in these 
grades are an intended audience. 

D. Background Foundation, and Theoretical Framework: 

The research literature on effective teaching from the past 10 years indicates the importance of 
classroom conditions that depend directly on the ability of teachers to organize and manage their 
classrooms. Some of these conditions include the productive use of time (Frederick & Walberg, 1980; 
Stallings, 1980); student attention or involvement in learning activities, task-oriented and goal- 
directed environments; and opportunities to interact with the teacher and instructional activities of 
appropriate difficulty (Bloom, 1976; Brophy & Evertson, 1976; Fisher, Berliner, Fdby, Marliave, 
Cahen, & Dishaw, 1980; Medley, 1977). Kounin's (1970) pioneering work on student work involvement and 
deviancy identified dimensions of teacher management behaviors that laid the groundwork for further 
studies on classroom management and organization. 

The Classroom Organization and Management Program (COMP), which drew its theoretical base from 
Kounin's work, was developed from a programmatic series of descriptive, correlational, and 
experimental research studies designed to discover key management practices and strategies and to 
test these in field experiments conducted with teachers in their own classrooms (Emmer, Sanford, 
Clements, & Martin, 1983; Evertson, Emmer, Sanford, & Cements, 1983; Evertson, 1985; 1988). These 
studies, funded by the National Institute of Education and conducted through the Research & 
Development Center for Teacher Education, the University of Texas, Austin, and the Arkansas 
Department of General Education, Little Rock, Arkansas, formed the basis for the Classroom 
Organization and Management Program. The research was conducted in three phases - descriptive, 
correlational, and experimental, each of which is described below. 



Phase One included two descriptive/correlational studies (Studies 1 & 2 in table 1) in elementary 
and secondary classrooms. In these studies, observers conducted extensive observations at the 
beginning of school, starting on the first day and continuing throughout the year. At the end of the 
study, investigators correlated teachers' management practices with student outcomes such as task 
engagement, inappropriate and disruptive behavior, and student attitudes and achievement. Phase One 
demonstrated that there were significant correlations between specific teacher practices and student 
behaviors, both academic and social. The large number of teacher behavior and student outcome 
variables made it possible to examine patterns of teaching and to rank teachers in order of their 
management effectiveness and their effects on student outcomes (Emmer, Evertson, & Anderson, 1980; 
Evertson & Emmer, 1982). Data from Phase One provided the content for workshop materials and teacher 
manuals to be tested in Phase Two (Evertson, Emmer, Clements, Sanford, Worsham & Williams, 1981; 
Emmer, Evertson, Sanford, Clements, & Worsham, 1982). 

Phase Two involved two experimental field studies (Studies 3 & 4 in Table 1) conducted to test 
the effectiveness of the information in the manuals on teachers' management behaviors at the 
beginning of the school year. Phase Two demonstrated that teachers in the experimental groups not 
only used management strategies and procedures significantly more than the control groups, but also 
students in these classrooms had significantly higher task engagement, less inappropriate and 
disruptive behavior, and higher academic success (Evertson, Emmer, Sanford, & Clements, 1983; Emmer, 
Sanford, Clements, & Martin, 1983). In Phase Two, the experimental treatment was delivered by 
project staff at the Research & Development Center for Teacher Education, University of Texas, 
Austin. It became apparent after Phase Two that, while acquainting teachers with management 
principles seemed to be effective, other more efficient methods would have to be devised to export 
the training program more broadly. Was the classroom management training replicable in other sites? 

Phase Three resulted from concerns about how best to disseminate the program tested in Phase Two. 
The goal of Phase Three was to determine if school district personnel, all of whom had had classroom 
experience, could deliver the manual-based training to teachers within their schools. In Phase 
Three, three Arkansas school districts participated ic two field-based studies (Studies 5 & 6 in 
Table 1) designed to test the effectiveness of t'-r classroom management training using school 
district personnel as trainers and observers. Findings from Phase Three showed that, even with 
different trainers and with considerably less observation time available than in Phases One and Two, 
this triiner-of-trainers of teachers approach produced results similar to those in Phase Two. 
Teachers in the experimental groups in these studies used the management practices from the training 
significantly more than the control group and had significantly less off-task behavior and less 
disruptive and inappropriate behavior (Evertson, 1985; in press). Comparisons of results across 
Studies 3, 4, 5, and 6 are shown in Table 5. 

In all, two descriptive/correlational studies were conducted from 1977«79, involving 28 elementary 
and 102 secondary classrooms (two classrooms each for 51 teachers); four experimental field studies 
were conducted 1980-83, involving 70 elementary and 54 secondary classrooms (including grades 1-9). 
Table 1 gives a chronology of the research studies, grade levels, total hours of in-class 
observation, and students and teachers involved. 

Results from these studies indicated that recommendations and suggestions for teachers that are aimed 
at planning rules and procedures ahead of time, presenting these to students along with expectations 
for appropriate behavior, and providing regular feedback to students about academics and behavior can 
result in improved task engagement. In addition, the experimental groups in these studies showed 
less inappropriate student behavior, smoother pacing of instructional activities, higher engagement 
in academic activities, improved teacher monitoring of student work, more efficient transitions 
between activities, and a more task-oriented focus when compared with a control group without such 
training. 
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Table 1. DeecHptlve/correletlonet, experimental, and eapeMaiental/aveluation etudtea conducted to 
investigate effectlvaclaasroosi stanaoeaent pncUcea. _^ - ^ =a===t 





Yra. 


Grades 


Subja 


Obe. 
Hrs. 


Sen Is 


Teach- 
era 


Class- 
rooms 


Its. 


SOURCES 


beaeHoHve/Corretational Studies 
1 Classroom 
Organisation 1977-78 
Study 
(COS) 


2-4 


Rdg/siath 


900* 


8 


28 


28 


650 


Emner, Evertson, 
8 Anderson 
(1960); Evertson 
8 Anderson (1979) 


2 Jr. High Classroom 
Organization Study 
(JHCOS) 


1978-79 


7-8 


Eng/meth 


1,400 


11 


51 


102 


2,800 


Evertson t Emmer, 
(1982: 


£vDcrfmantaL Field Studies 
3 Classroom Management 

Improvement Study 1980-81 
(CMS) 


1-6 


LA/ddg/ 


910 


H 


41 


41 


1,066 


Evertson, Emmer, 
Sanford, I 
Clements s (1981) 


4 Jr. High Management 
Improvement Study 
(JHM1S) 


1981-82 


*v el 

7-8 


Cng/math 




1 u 


JO 


76 


2,052 


Emmer f Sanford, 
Clements, t 
Martin (1983) 


Experimental /Evaluation 
5 Classroom Management 
Training for 
Elementary Teachers 

(CMTET % , 


1982-83 


1 

1 


bAr rang 

math 


176 


3 


29 


29 


725 


Evertson 
(in press) 


6 Cla'troom Management 
Training for 
Secondary Teachers 

(CMTST) 


1982-83 


7-9 


Eng/math 


96 


J. 


16 


16 


?8* 


Evertson (1985) 


















Totals 






3, AOS 


29 


203 


422 


7,677 





Features 1 How the nrooram operates : 

1. Scope : The Classroom Organization and Management Program (COMP) is intended to supplement other 
professional development activities and provides the necessary foundational management skills on 

which other academic and instructional programs must build. 

2. Curriculum and instruction approach : COMP provides teachers with management ideas and materials 
and involves them in activities directly relating these to classroom management in their own 
classrooms. The program is presented with a tripartite focus on pawning implementing, and 

mflmtaininr- 

3. Learner activities : COMP is designed to be an inquiry-based approach to professional develop- 
ment. The workshops are models of a process that can be implemented in the school's own profes- 
sional development program. Outline of workshop activities and participants roles is shown 

below. 

a. Assessment and problem identification 

Teachers begin by using focusing checklists to assess their own classrooms and to identify 
areas of concern. The group leader helps relate concerns to relevant research. 
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Activities engage participants in examinations and analysis of the research-based principles, 
using vignettes, cases studies, film, and simulations. This cycle is repeated for each topic 
and the instructional process for participants and group leaders includes the following: 

1. Presentations based on research findings (program staff/group leader) 

2. Group discussion and problem-solving using guidelines, checklists, and case studies (small 
groups or teacher/administrator teams) 

3. Identification of guidelines and approaches applicable to classrooms 

4. Discussion of strategies to be implemented (small groups or teacher/administrator teams) 

5. Feedback on tentative solutions to problems (total group and small groups) 

6. Individual group reports on tentative solutions (small group/team reporters) 

7. Sharing of strategies, and techniques (total group) 

c Formulation of implementation plans 

1. Reexamination of diagnostic assessments (individuals) 

2. Focus on problems participants identify (total group) 

3. Formulation of plans for new approaches (individuals, small groups, or teacher/administra- 
tor teams) 

4. Follow-up on roles, responsibilities, and tasks (teacher/administrator teams) 

(e.g., arrangements for peer observations, feedback on practice of new techniques) 

5. Development of structure for ongoing follow-up and maintenance 
of plans (group leader, small groups, teacher/administrator teams) 

6. Development of structure and process to continue inquiry in school settings (small 
groups, teacher/administrator teams) 

4. Learning materials; Materials used in the professional development workshops include two 

commercially published books, Classroom Management for Elementar y Teachers (Evertson et al., 
1989) and Classroom Management for Secondary Teachers (Emmer et al., 1989), a workshop manual 
for teachers, and a manual for trainers. Both manuals include six modules: 

a. Orgnnhring the classroom Teachers examine research-based findings on effective classroom 
arrangement and apply those findings to their own classrooms. 

b. Planning and teaching rules and procedures Teachers decide what behaviors are acceptable and 
unacceptable in their classrooms. They then consider what procedures students must follow in 
orde* to successfully participate in class activities, to learn, and to function effectively 

in ihe school environment. Finally, they learn how to teach these needed procedures. 

c* Student accountability Teachers examine effective procedures for keeping snidery responsible 
for their work and then develop accountability procedures for their own classrooms. 

d* Consequences and incentives Teachers examine guidelines for establishing consequences and 
incentives, examine those proven useful in the classroom, and develop consequences and 
incentives for their own classrooms. 

c Planning and organizing instruction Teachers examine ways to organize instruction to provide 
learning activities at suitable le"jl* for all students in their classes. 

f. Conducting instruction and maintaining momentum Teachers examine ways to conduct clear 
instruction that includes all students and keeps the lesson moving forward. 
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5. Staff activities and staffing patterns; A district may choose to have their own personnel 
trained to deliver workshops to their teachers. Time for the workshops and for follow-up with 
teachers will need to be integrated into the school calendar. A coordinator to oversee or 
deliver workshops and provide assistance to participants is needed. 

6. Staff development activities: The Classroom Organization and Management Program may be dissemi- 
nated in two ways: 

a. Training Teachers Teachers attend a three-day workshop, receive materials and training, 
and directly relate this to their own classrooms. There is a 1 to 1 1/2 day follow-up which 
is optional, however, most program adopters find the follow-up beneficial. Training takes 
place on-site. Maximum number of participants in workshops is 20. 

b. Training Twiners of Teachers Selected teachers or administrators from a school or district 
attend a seven-day workshop and receive materials and training to enable them to provide the 
Trainer-of-Teachers model to classroom teachers. There is a 1 1/2 to 2 day follow-up which is 
optional. Training can take place in Nashville or program staff members can come to an 
appropriate site away from Nashville. Maximum number of participants for this model is 12. 

7. Management activities: There are no specific management activities required; however, 
arrangements must be made for teachers to be away from their classrooms. Scheduling of workshops 
and opportunities for follow-up in classrooms of teachers who want feedback should take place. 
Some release time for teachers to observe each other should also be scheduled. 

8. Monitoring and evaluation procedures: Participants are taught observation techniques that they 
can use to provide feedback to each other. Also, trainers in the Trainer-of-Teachers model are 
taught additional observation techniques. 

F. Significance of program design as compared to sim ilar programs : 

Teachers are provided with research-based principles for managing and organizing classroom tasks. 
With these principles they plan and develop strategies that directly relate to their own classrooms 
and needs. Teachers can pinpoint their own trouble spots through self-assessments and through peer 
observations as the year goes on. The program is unique in that it deals directly with teachers' 
management concerns and provides a structure for them to use in problem solving and develop a 
decision making frameworks for their own classrooms. Few other programs combine research-based 
principles and utilize teachers' own concerns as a basis for individualizing management procedures 
in specific classrooms. 



111. POTENTIAL FOR REPLICATION 

A. Settings and participants (Development and Evaluation sites): 

COMP was originally tested with. 124 teachers in 29 schools that varied in size from rural to 
metropolitan in two states (See Table 1). Student and teacher populations represented are black, 
white, and hispanic Ability to incorporate the processes taught in the workshops was accomplished 
equally well in all of the schools, suggesting that the processes are can be applied in a wide range 
of schools and student populations. 

In comparing results of Studies 5 & 6 with those of Studies 3 &. 4, 10 variables were significant for 
the experimental group in all four experimental studies. Seven additional variables were 
significant b the same direction in 3 out of the 4 studies. In no case were trends reversed from 
one study to another. (See Table 5). Treatment effects were always in the same direction. In a 
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few cases the same variables were sot measured across all studies. The findings for the 
experimental groups were quite strong in all four experimental studies. Teachers trained by 
district personnel performed about same or better as those trained by researchers. 

Following Studies 5 and 6, the Arkansas Department of General Education implemented the program 
statewide and designated a state facilitator for adopters. Personnel from the original group of 
trainers served as state and local trainers. From 1963 until the present, 318 of the then 333 
school districts in Arkansas have implemented the program. Facilitators at the Arkansas Department 
of General Education have reported that the program is seen very positively and is highly popular 
with school districts and teachers. 

The Training-of-Trainers model was implemented in Greeley, Colorado (Weld Co„ Dist # 6) in 
response to the district's need to develop a classroom management program for all of their teachers 
Fourteen teachers on special assignment took part in a workshop conducted by Dr. Evertson in 
October, 1987. Eleven teachers in this group and one adminir'rator have since provided trainin g for 
160 of the district's 700 teachers. Ron Roggy, Staff Development Coordinator, has reported a 
waiting list of between 35 to 50 teachers. Response from teachers in the workshops has been 
extremely positive. Evaluations have ranged from 4.5 to 5.0 on » five point scale of satisfaction 
with the material and evaluations of its usefulness. Teachers and administrators report smoother 
starts at the beginning of the year. 

A second site that implemented the Training-of-Trainers model was Kentville, Nova Scotia. This 
district set aside funds to adopt the program and to have 11 volunteer teachers trained to be 
district trainers. This training was completed in October, 1987, and workshops for 60 teachers have 
been provided since September, 1988. Greg Ross, Eastern Area Supervisor, reports a waiting list of 
teachers asking to be in the workshops. Teachers' and administrators' responses as well as 
responses from the school board and the teachers' union have been extremely positive. 

The Association for Supervision and Curriculum Development (ASCD) released a film in 1988 based on 
COMP in 1988 entitled Classroom Manay^nr with Pat Wolfe. 

B. Rcplicable exponents "d Documentation: 

All elements of the program are transportable. The materials for teachers and contains a manual 
and one or two texts, depending upon grade level (elementary or secondary). Materials for potential 
trainers include a teacher's manual, two texts, a trainer's manual, and materials for conducting 
observations. No special equipment is needed. 

C. User foailkgjaejtt; 

Model 1 - Traming-of-Teachers: Teacher training requires release time for teachers to attend a 
three-day workshop conducted by the program staff and the assignment of a local coordinator to 
coordinate visits to teachers' classrooms where follow-up and peer support is requested. Materials 
must be purchased for teachers. Principals' and other administrators' attendance at the workshop is 
highly desirable. 

Model 2 - Tranuag-of-Traxnen: Training of on-site trainers requires release time for teachers or 
district personnel for about seven days total to take part in workshop training, peer obsc:\^j 
training, and practice with workshop materials, and content Materials must be pur chased for 
trainers; these include a trainer's manual, teacher's manual, and two commercially published texts. 
Principals' and other administrators' workshop attendance and involvement is highly desirable. 
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D. Co*ts (for implementation and operation) 

Projected costs for both the Tramer-of-Teachers and the Trainer-of-Trainers models and for both 
beginning and subsequent years are explained in Table 1 Costs will vary depending upon the numbers 
of groups of teachers to be trained and where training is done. For example, it would be possible 
for potential trainers to participate in training workshops conducted in Nashville as well as for 
project staff to conduct training at other sites* 



Table 2. Projected costs of the Clsssroon Orgenfietlon and Msnsgement Progru 



Stsrt-uo 
Tftr 1 



flnmiiap 
Itic * and 



Personnel 



Training 



Modal 1 
Teacher Training 
(Max: 20) 

Release time for 
teachers to attend 
3* day workshop 



Honorarium ♦ expenses 
for program staff 
trainer <S750/4ay) 



Nodal 2 

Trainer Training 
(Max: 12) 

Release time for 
district personnel/ 
teachers to attend 
7 days of workshop 

Honorarium ♦ expenses 
for program staff 
trainer (S8S0/day) 



Model 1 
Teacher Training 



Model 2 
Trainer Training 



Coordinator's time for scheduling 
for scheduling follow-ups or for 
training 



(optional) 



Equipment 

Materials & 
supplies 

Other 



Total Cost 
Cost per user 



Overhead projector and VCR/monitor 
% 50 S100 



Travel t expenses for Travel I expenses for 
one program staff one program staff 



same 



trainer (1 trip) 

$3,250 
(plus expenses) 



S 162 
(plus expenses) 



trainer (2 trips) 

S6,450 
(plus expenses) 

$ 595 
(plus expenses) 



Travel & expenses for one 
program staff trainer 
(1 trip) optional 



(plus expenses) 



IV. Evidence for riaitm Statement* 

A. riaim Typ» 1: Academic Achievement - Changes in Knowledge and Skills. 

Acquisition of Factual Knowledge: In Study 6 (CMTST), students in 6 experimental classrooms in 
grades 7-9 made greater gains on standardized tests in language arts, reading, and mathematics 
than did students in the control group classes. 

B. Description of Methodology frr Haim 1* 

1. Desig n: To assess COMFs possible effects on student achievement, student achievement test 
scores (math and language arts) taken the prior spring were used as pretests for both the 
experimental and control group classes. Achievement test scores for the same students taken in 
the following spring, after the workshops were completed, served as post-tests. 

Z Sample : The sample includes 272 students in English and math classes (grades 7-9) who had valid 
pretest and posttest scores, an average of 17 students per class (range 12-24). Math classes 
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were tested for effects on math subtests of the achievement tests, tad English classes were 
tested for effects on language arts and/or reading scores. Of the 272 students, 164 were in 
English classes and 106 were in math classes; 126 students were in experimental group cla sses and 
146 in control group classes. The sample contained 106 7th graders, 70 8th graders, and 94 9th 
graders. This sample of students represented approximately 60% of all students in grades in 7-9 
these two subjects. 

3. Instruments and nrnegdurwi The tests used included the Stanford Research Associates Test (SRA) 
and two criterion-referenced tests, the district's Criterion-Referenced Test (CRT) and the State 
Assessment Test of Basic Skills (SATBS). Split-half reliability for the SATBS was .93. A study 
of the CRTs relationship with the SATBS was conducted by the district and showed a correlation 
of .90 between the two tests. Statistical analyses procedures are described in detail in 
Evertson, Weade, Green & Crawford (1985). 

Each row in Table 3 represents available data for a single class. The CRT analysis in language 
arts consisted of two (2) experimental classes and four (4) control classes. The CRT math 
analyses compared two (2) experimental and two (2) control classes. For the rem aining six (6) 
classes, prescores on the SATBS were examined; all were 9th grade classes. Two pairs of classes 
were chosen for class-by-class comparisons. The selection was based on determination of which of 
the two available experimental classes had prescores that were closest (based on the mean) to the 
piescore of the control group class in the appropriate subject area. This design and the 
attendant analyses allowed the use of 14 of the 16 da&vs. Two experimental classes remain 
unanalyzed because there were no appropriate comparisons. Scores on the CRT and SATBS reflected 
a percentage of objectives mastered. The SRA test provides nationally normed percentiles; these 
were transformed into equal interval normal curve equivalent (NCE) scores for inferential 
analyses. 



Tablt 3. Experimental Oeaign: Matching of Pre- and Poatteata 



Preteit Meaaur.a 



Poatteat MMiurn 



1982 
Math 



CRT 
LA 

E 
E 
C 



1982 
Math 



SATSS 
Reading 



1983 CRT 
Math 



LA 



E 
E 



C 
C 

E 
E 



C 
C 



(E) 
E* 
C* 



E > Experimental group 
C « Control group 



(E) 
E* 
C* 



1983 SRA 
Math Raading 



(E) 
E* 
C* 



(E) 
E* 
C+ 



() Thaaa elaaaaa wara not analyzad becauaa thara vara no appropriate eoapariaona. 
•: Thaaa eiaaaaa wara uaad in elaaa-by-eiaaa eoapariaona baeauaa thair pra-aeoroa 
in raading Mtehad eioaaly. 

Thaaa two eiaaaaa ware uaad in the aath eiaaa-by-elaaa eoapariaona baeauaa thair 
preacoroa Mtehad eioaaly. 
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4. Pat* cnllectien; The test data were collected as a regular part of the school districts 1 testing 
program. Standardized procedures were used in all classrooms. The CRTs were collected and 
scored electronically in the district's central offices. The SATBS were sent to the Arkansas 
Department of Education's Division of Curriculum and Assessment for scoring. The SRA's were 
scored by Stanford Research Associates. 

5. Data analysis : Three approaches were used to analyze achievement data for the 14 classes. They 
included (1) analysis of student raw gains and regression adjusted performance (ANCOVA) without 
regard to classrooms, (2) analyses of between-class variance on raw gain and regression-adjusted 
gain (ANCOVA), and (3) paired comparisons, class-by-class, of the two pairs of classes with SATBS 
pre-scores and SRA post-scores (ANCOVA and, for reading and language arts, multivariate ANCOVA). 
However, analyses 3 is based on only two classrooms and is, therefore, likely to yield unstable 
estimates. 

6. Description of results : For analysis 1, raw gains on the CRT (10 classrooms, students 1 scores 
"pooled" without regard to classes) revealed differences in favor of the trained teachers 9 
students (language ^ £ (1,116) * 32.82, p < .0001; effect size * 1.13; math: £ (1,66) - 4.26, 
E ■ .043; effect size « .59). For analysis 2, between-class CRT raw gains also favored trained 
teachers (f m (1,116) - 479.71, p < .0001; math: E (1,66) « 110.04, B <.0001). 

In all, 11 comparisons were generated: 9 showed higher means for the trained teachers, and 7 of 
these 9 were statistically significant (g < .05). Neither of the two comparisons favoring the 
control group were significant. These findings are summarized in Table 4. Full discussion of 
the statistical procedures is available in Evertson, Wcade, Green & Crawford (1985). 



Table 4 Summary of Achievement Analyses 



CRT Scores Across Classes 
Raw Gains: 

Language Arte 
Math 

ANCOVA • Adjusted gains: 
Language Arts 
Math 

CRT Scores. Setyten-Cl iss Analysis 
Raw Gains: 

Language Arts 
Math 

ANCOVA * Adjusted gains: 
Language Arts 
Math 

Paired Class»bv«Claas Comparisons 
ANCOVA - Adjusted gains: 

Language Arts 

Reading 

Math 
• * p < .05 



Exq. Grouo 

Higher * 
Higher • 

Higher • 
Higher (p «.U> 



Higher * 

Higher * 

Higher • 

Higher • 



Higher (ns) 



Con. Group 



Higher (ns) 
Higher (ns) 
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C. Claim Type * Improvements in Teachers' Attitudes and Behaviors 

Change m Teacher Behavior. Teachers participating in the workshops used the effective managerial 
and instructional practices and principles provided in the workshops significantly more so than did 
the control teachers. 

D. Description of Methodology for Claim 2: 

1. Efiaiffi- Four experimental field studies were conducted to test the effects of workshop training 

in principles of classroom management and organization. An expenmental • matched control group 
design was used and observational data were collected in all classrooms beginning with the first 
day of school and continuing throughout the year in Studies 3 & 4 and until November in Studies 5 
& 6 (Table 1). Hours of observation and subject matter areas observed are shown in Table 1. 

2. Samfik: Samples included 37 experimental and 33 control group teachers (Grades K-6) and 26 
experimental and 28 control group teachers (Grades 7*9). In Study 4 subject areas were math, 
science, English, and social studies; in Study 6, English and math. In Studies 3 & 5 observers 
saw primarily readbg and mathematics, but occasionally other lessons were also observed. 

3. Instruments and procedures: A variety of observational measures was used. These included narra- 
tive field notes, classroom component rating, of lesson management, inappropriate and disruptive 
behavior, efficiency of instructional routines, percentages of students engaged, records of time 
use, efficiency of transitions, feedback to students, etc. Trained observers were used in all 
studies. Training activities included reliability checks and practice with videotapes of 

classroom instruction. Observers collected narrative records in all studies and in Studies 5 & 
6, verbatim audiotapes of class lessons were collected. Observers completed classroom ratings 
(5-point scales) after each observation and summary ratings at the end of the studies. Observers 
were trained to a reliability criterion of .85 on classroom rating scales and other measures. 
Regular reliability checks were conducted as data collection progressed to prevent observer 
drift. Frequency counts of on* and off-task behavior were reliable at .85. 

4. Data collection: Workshops were delivered prior to the opening of school to all of the teachers 
in the experimental groups. AH teachers in both groups were observed on either the first or 
second day of school and in Studies 3 & 4 emphasis was given to the first 8 weeks. Workshop 
teachers were asked not to share materials or to discuss the training with teachers in the 
control group. 

5. Data analysis: At the end of data collection, mean scores for the classroom rating scales were 
computed and F-tested using one-way analyses of variance. Narrative records were also scored 
using reader ratings (5-point scales) to assess the degree to which teachers used the prescribed 
practices. Table 5 shows the findings for each of the key classroom management variables for 
each of the four experimental studies. 

A significance level of p <. .05 indicated an effect size of 30 or larger. Effect sizes could 
not be computed for Study 4, however, because standard deviations are not reported; however, it 
seems reasonable to assume that the practical significance of the statistical differences is 
similar to that of the other studies. 

6. Descriptio n of results : In Table 5, a plus (+) indicates that the experimental group had 
significantly higher scores than the control group on the variable. A minus (•) indicates that 

the experimental group had significantly lower scores than the control group on the variable. Of 
the 32 key variables listed in Table 5, 59% (19 variables) were significant in favor of the 
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Table 5. Results fro* obstrvatfonil ratings colltettd In experimental fftld studies of 
implementation of classroom Management procedures. 



Teacher Behavior Variables 
1. Readying the Claaarooai 

Classroom is ready for school 

II. Developing lules end Procedures 

a. Efficient administrative routines 

b. Appropriate general procedures 

c. Efficient small group procedures 

III. monitoring 

a. Of student behavior 

b. of transitions between activities 

IV. managing Student lehavior 

a. Rewards good performance 

b. Consistent management of behavior 

c. Signals appropriate behavior 

d. Allows class to get out of hand 

e. Stops disruptive behavior quickly 

f. Stops inappropriate behavior quickly 

g. Ignores inappropriate behavior 

V. Teaching Rules I Procedures 
(First neek of aehool) 



Study 3 

CNIS 
fir 1-6 

N»41 



♦ 

NS 



♦ 



♦ 

NM 

♦ 



Study 4 

JMIS 
fir 6*8 

N»38 



NS 



Study 5 
CNTET 

fir 1-6 
N>29 



NM 



♦ 



♦ 
♦ 

NS 
♦ 



♦ 



♦ 
♦ 
♦ 

NS 
NS 
♦ 

NS 



Study 6 
CMTST 

fir 7-9 
N«16 



NS 
♦ 

NM 



♦ 
♦ 



♦ 
♦ 
♦ 

♦ 
♦ 





a. Presents, reviews, discusses rules/procedures 


♦ 


<♦) 


♦ 


NS 




b. Rehearsal, practice included in presentation 


♦ 


<♦) 


♦ 


♦ 




c. Teacher stays in charge of class 




NS 


♦ 


NS 


VI. 


Organizing Instruction 










a. Student ettention spans considered 


NS 


<♦) 


♦ 






b. Appropriate lesson pacing 


NS 


NS 


♦ 


♦ 




c. Lessons related to students backgrounds 












and interests 


NH 


NM 


NS 


♦ 




d. Low amount of down time when students 












are waiting 


NM 


NS 


♦ 


<♦) 


VII. 


Student Accountability 












a. Checks students 1 understanding 


♦ 




♦ 


<♦) 




b. Consistency in enforcing academic work 












standards 


♦ 




♦ 


♦ 




c. Suitable routines for collecting and 












checking work 


♦ 


♦ 


♦ 


♦ 




d« Teacher holds students responsible for 












academic work 


♦ 




♦ 


NH 




e. Monitors progress in completing assignments 


♦ 




NM 


NM 




f. Class has task-oriented focus 


NS 


NS 


♦ 


♦ 




g. Teacher plans approp/iate amount of class work 


♦ 


♦ 


♦ 




VIII 


. Instructional Clarity 












a. Describes objectives clearly 


♦ 


NS 


♦ 






b. Clear directions for assignments 


♦ 


<♦) 


♦ 


♦ 




c. Clear explanations 4 presentations 


♦ 


NS 


♦ 






d. Checks students 1 understanding of instructions 


♦ 


♦ 


♦ 


♦ 



♦ Experimental group had significantly higher mean scores; p < .05 

* Experimental group had significantly lower mean scores; p <~.05 
<+) or <•): positive or negative trend; p < .10 

NM: variable not measured in this study 
NS: not significant 



13 



R. n.im TW 3: ImprovemeflU in Students' Attitude »nd Behavior* 



Change m Student Bdutnor. In classrooms where teachers used the management and instructional 
principles provided in the workshops, student task engagement was higher, student off-task behavior 
was lower, and inappropriate student behavior ws $ lower. 

F. Description of Methodology far n»im V 
1. Design: (Same as in Claim 2) 

ISmpJfi: (Same as in Claim 2) Each class averaged 18-30 students, who were observed as they 
participated in their classes. 

3. Instruments and procedures; The instruments and procedures are the same as those described u 

Claim 2. In addition frequency counts student engagement in tasks (% of on- and off-task 
behavior), ratings of inappropriate and disruptive behavior, student success in lessons, student 
attempts to get help on assignments, student cooperation in classroom tasks were recorded by 
observers. Percentages of students engaged were recorded a minimum 0 f four times per hour of 
observation, yielding up to 100 estimates of student engagement over the course of the studies. 
Narrative records used in all studies included descriptions of student behavior as well as 
teachers' practices. Observers completed classroom ratings (5-point scales) of these variables 
after each observation and summary ratings at the end of the studies. Observers were trained to 
a reliability criterion of .85 on classroom rating scales and percentages of students engaged 
Regular reliability checks were conducted as data collection progressed to prevent observer 
drift. 

4. Data collection: Observers were t/ained to collect data on student engagement, cooperation in 

classroom tasks, behavior in class lessons, and general behavior in class during each 
observation. These variables were measured as class observations were made (described in Claim 
2). 

5. Data analysis; At the end of data collection, mean scores for the classroom rating scales 

measuring student outcomes were computed and F-tested using one-way analyses of variance. 
Percentages of classroom of students engaged vs. not engaged in classroom tasks were calculated. 
Narrative records were also scored using reader ratings (5-point scales) to assess the degree to 
which students cooperated in class activities. Table 6 shows the findings for the student 
variables in each of the four experimental studies. 

A si gnifi cance level of p <. .05 indicated an effect size of 50 or larger. Effect sizes could 
not be computed for Study 4, because standard deviations are not reported, however, it seems 
reasonable to assume that the practical significance of the statistical differences is similar to 
that of the other studies* 

6. Desmpdon of results: In Tab<e 6, a plus (+) indicates that students in the experimental 
group had significantly higher scores than the control group on the variable. A minus (•) 
indicates that students as a group the experimental group classes had significantly lower ratings 
than the control group on the variable. 
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Table 6. Ittults from Observational Data Colltettd In ExpariMntal Fitld studUt of 
ttudant Ichavfer. 



Study 3 Study 4 Study 5 Study 6 

CMIS JMIS CMTET CMTST 

Gr 1-6 6r 6-8 Cr 1-6 Cr 7-9 

M«41 N>38 N>29 M16 



Student VarUblat 

a. Sts. call-out fraqutntly • us 

b. sts* frequently out of aeata NM - f-> 



c. Low amount of inappropriate 

behavior nh NS 



<e$ .39) 



d. X of atudenta on-task ♦ ♦ <♦) 

(es .48) 

e. X of students off-task 

f. Student aueeesa in elaaa lessons NS <♦) (♦) 

<ea .40) 



♦ : Experimental group had significantly higher mean scores; p < .05 

• : Experimental group had signi f icantly lower mean scores; p <"\05 
CO or (-): positive or negative trend; p_< .10 

es: Effeet size 

MM: variable not measured in this study 
MS: not significant 



G. Summary of supplementary evidence for each claim: 

Claim V 

Micro-anaiyses of class lessons in Study 6 (Evertson & Weade, in press) and Weade & Evertson (1988) 
have shown differences in the quality with which the experimental group and control group teachers 
delivered classroom lessons. The findings from the micro-analyses lend support to the connection 
between effectively managed classrooms and student achievement gain. Key differences between 
experimental and control group teachers were in the provision of opportunities for students to 
engage in academic tasks. Also, teachers in the experimental classes provided better signals, 
protected lessons from interruptions and diversions, and held students accountable for participation 
in lessons. 

Qaim 2; 

Reports from teachers who have experienced the workshops in Boulder and Greeley, CO have indicated 
that teachers have changed their classroom practices and these changes have resulted in smoother 
starts at the beginning of school. Ross Beck, Superintendent, Texarkana, AR, school district has 
written: Teachers continue to be impressed with the program and use the workbooks^ reference 
material. As we have continued the program since its inception, the overwhelming comment received 
from teachers is that the beginning of school is so much easier. It is easy to walk into a 
classroom and notice who has and who has not had the training. This is most obvious when looking 
for signs of rules and procedures having or not having been taught." 

Beck's comments also lend support for Claim 3. He states, "Over the last 5 to 6 years, we have 
noticed a sharp decrease in the use of corporal punishment. It is safe to say that the use of 
corporal punishment has decrease at least 50% in the junior high schools since the inception of the 
Classroom Organization and Management Program. I attribute this to teachers 1 actively seeking more 
productive alternatives in working with students." 
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H. Internretation and dUcinsion nf rtsults: 



1. Relationship between effect and treatment: The elements of the training were directly measured 

in the observations. For sample, the measures listed in Table 5 are grouped into sections 
relating to the content covered in the workshops. For example, SflldCflt Acc ountability is covered 
in workshop Module 3 and is directly measured by the set of variables in Table 4 under that 
section. The measures shown in this table were developed to directly assess teachers 1 use of the 
material and principles taught in the workshops. 

2. Control of rival hypotheses: 

a. Control for syst ematic bias in selection of subjects for experimental and control groups. 

Systematic bias could have potentially been a threat to validity of the results in at least 
two ways. In the first way, teachers with reputations as better managers or with more 
experience could have been systematically assigned to the experimental group. To control for 
this, teachers in all studies were matched on experience, grade level, subject area, and other 
key demographic variables then randomly assigned to the training and control groups. 

A second potential threat to validity was that control teachers 1 classrooms might have had 
lower ability students or special ne&'s students to a greater extent than the experimental 
teachers 1 classrooms. If this were so, teachers in these dassr x>ms would have more 
d Jiculty keeping students engaged and managing student behavior. Thk could then have 
resulted in lower frequencies of on-task behavior and higher ratings of inappropriate and 
disruptive behavior. Thfc. possibility was addressed in all of the experimental studies 
(Studies 3 • 6). With one exception, only teachers whose classes were composed of typical or 
average ability students were included. Gifted or special education resource classrooms were 
not included in the studies. In Study 5, however, some teachers did include some teachers who 
teach classes of low achieving students. In this case, treatment and control groups were 
balanced for student entering achievement level as well as teaching experience, grade level, 
and other demographic variables that could affect the outcome of the study. Treatment and 
control group differences for teachers of lower achieving classes showed the same pattern as 
those the average achieving classes. That is, teachers of low achievers were also able to 
benefit from the workshops in comparison to the control group teachers. 

b. Halo effects: The possibility that halo effects could have resulted in observers rating warm, 
friendly, or charismatic teachers higher on ottar key management variables was also addressed. 
Variables that are particularly susceptible to positive or negative halo effects (relaxed, 
pleasant atmosphere; uses listening skills; and expresses feelings) shored no significant 
difference between experimental and control groups in any of the studies. Teacher enthusiasm 
was significantly higher in the experimental group in only one study (Study 4). Also, at 

least two observers saw all teachers in all of the studies. In the larger studies (Studies 3 
& 4), regular reliability checks were conducted every four to six weeks by members of the 
project staff who had not previously ; ;en the teachers. In addition, observers were not aware 
of who was in the experimental or control groups in any of the studies. Teachers were also 
asked not to talk about their participation in the workshops to observers about the studies. 
From the information we have, this request was honored. 

c Hawthorne effects! It is possible that the Hawthorne effects could be operating in the 
classes of teachers who were in the experimental group simply by their having been in the 
workshops. However, the purposes of the experiments were told to both groups; treatment of 
the experimental and control groups differed only in that one group got the workshops. Both 
groups were observed; both groups received feedback at the end of the studies. Control group 
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teachers in Studies 3, 5 & 6 were promised and were included in workshops to the late spring 
after data collection. 

1. Educational SignjfigajQce, ff RfflffllfV 

1. Relationship of result* to needs : The original intent of this line of research and program 

development was to devise effective means of training teachers in effective classroom management 
strategies. The results of the studies conducted have shown the following: 1) There are 
strategies that, if used by teachers, can result in better student task engagement, more positive 
student behavior, and smoother instructional activities. 2) These strategies can be taught to 
teachers in relatively efficient ways. 3) School personnel can serve effectively as trainers and 
as on-site support for teachers as they learn and practice the principles and strategies. 4) 
And, in some cases, Lhere arc cot only effects on student behavior such as task engagement, 
inappropriate and disruptive behavior, but effects on student achievement as well 

2. Comparison of Results to Results fro m Other Programs : We know of no other programs designed to 

help teachers learn more effective ways of managing and organizing their classrooms that has the 
extensive research and development base of this one. 
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Classroom Organisation a Management ZxSS JUUk 

. Questions 

1. (Stt p. 17) Tha claim if atdt in tht submittal thtt thtrt art no other 
proartif likt this. Bow dots this program difftr from programs dtfsloptd by 
MtdtUnt Buntsr or Cirol Cunnings? 

The claim in the subnittal statement was that we knew of no other 
prograns with the research and development background of this one. The 
research studies on which this program is based have been published in the 
literature for the past ten years (60 articles and monographs in ERIC and 
CUE; plus two commercially published books) . other developers have used this 
research and have developed activities from it for parts of their programs. 
Since many states are now mandating courses in classroom management as 
necessary for teacher certification and school districts want it for inservice, 
these programs will probably increase, however, the research base used for many 
of them is this one. 

Madeline Hunter's program focuses on the essential elements of 
instruction. Teachers are trained to teach lessons using specific steps: to 
open with a lesson "set," to use but not abuse the principles of learning, to 
motivate the learning, to adjust the learning to the level of the learner, to 
come to closure after the instruction, etc. As such she does not deal 
specifically with organizing and managing the physical environment or student 
behavior. However, this program (COM?) and Madeline Hunter's program have been 
and are both being used side-by-side in many sites (South Carolina, Colorado, 
Arkansas, Texas). Hunter's program focuses more on instruction; this program 
focuses on the proactive setting up of the classroom: developing rules and 
procedures, teaching rules and procedures, managing student academic work, 
managing student behavior, and preparing to conduct instruction. 

Carol Cummings' book, Managing to Teach , focuses on some of the same 
principles as COMP; however, the first one third of her book is drawn from the 
research base of COMP. She uses this work in her workshops and gives us 
credit. 

3. Zs thtrt tny indicttion whtthtr this program is tore btntfieitl to schools 
serving tny ptrtieultr socio-economic ltvtl? Zs it truly generally applicable? 

The experimental studies wcr* conducted in schools that ranged from Title 
1/Chapter 1 to upper socio-economic status. Our data indicate that it is truly 
applicable to schools/classrooms across the full range of SES. It is true that 
often teachers in different schools face different problems and have different 
needs for skills and information; however, there appears to be something in the 
program for all participants. 

3. Zs thtrt tny tvidtnct thtt this program is bttttr than any othtr program 
thtt trits to totch classroom atntgtatnt and control, or is it sinply bttttr 
thtn no program at til? 

Ve have not conducted a study that compared our program directly with 
other programs that purport to teach classroom management, nor has anyone that 
we know of. However, indirect evidence that this program adds to teachers' 
skills even though they have participated in other programs comes from Studies 
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5 4 6. All 45 teachers in Studies 5 6 6 hid been trtined in Madeline Hunter's 
program. Therefore, the experimental and control group differences reflect the 
added skills the experimental group teachers gained over and above training in 
Hunter's program. 

4. Have any schools using this program dropped the program? 

Ve have asked this of many of the school districts that adopted the 
program. None of the school districts we talked with had dropped the program. 
The reason most give is that there are new teachers entering the system each 
year and, once trainers are in place, the program is easy and cheap to deliver. 
Teachers have the need, and notivation is high. In one school district, the 
program has been in place for 7 years and is a regular part of their beginning 
school staff development. 

5. How were the control group teachers selected? 

In all four studies, teachers were recruited to take part in the research. 
They were randomly assigned (blocking for years of experience and grade level) 
to treatment and control groups within buildings. Control group teachers were 
told the purpose of the studies and were promised a workshop after data 
collection was completed (late March to April). Ve had no problems getting 
volunteers for any of the studies. 

6. Did the control group get any training in classroom management at all 
during the project? 

The control groups received training at the conclusion of: each study, but 
not during the studies. 

7. (See p. 12, Claim type 2) rid the control group teachers even know that 
these principles and practices existed? 

Most of the principles and practices are those that most teachers are 
aware of and use to a greater or lesser extent. It was not that we were 
teaching them something they had never heard of, but that we were using the 
research findings to highlight the important elements in structuring the 
classroom environment. In so doing, the program provided a decision-making 
fra-aework, including example techniques and planning time, that could be used 
to apply to their own classrooms. It was, we believe, this framework that 
provided teachers a new way of looking at their teaching. As one teacher put 
it: "It's not that they make you use a different style to teach, but that, 
however you teach, you can incorporate the important principles. It isn't that 
they tell you the rules to have, but that you have to have rules." The 
teachers in the experimental groups in these studies for the most part did not 
have to focus on getting control by using various disciplinary interventions 
(assertive discipline, etc.) because they gained control from the beginning. 

8. Please define "effect size" and describe how it is calculated? 

Effect sizes were calculated by subtracting the control group means from 
the experimental group means and dividing by the standard deviation of the 
control group. Ve used it to indicate the practical significance of a finding 
whether or not there was statistical significance. 
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9. (See Tables 3 6 4) low many teachers were involved in the evaluation of 
Claim 1? 

The achievement scores available for the 16 classes were those that the 
school district already had. Students had not all been pre and post tested 
with the same instruments because they were at different grade levels. 

Ten of the 16 classes were measured the same pre- and post-tests. Two of 
the 16 were not analyzed because of lack of an appropriate control. Four were 
used in class-by-class comparisons. In all, 14 teachers were involved in the 
evaluation of Claim 1. 

10. (Set p. 13) Please provide a summary statement for Table 5. 

This table shows the mean scores from the observational data for key 
teacher behavior and teaching practice variables across the four experimental 
studies. The variables listed under each section correspond to the key 
principles in the training workshops. This table shows that for most of the 
key variables the experimental group teachers mean scores exceeded those of the 
control group teachers. 

11. (See p. 15) Please provide a summary statement for Table 6. 

This table shows the mean scores from observational data for key student 
behavior data across the four experimental studies. Student engagement 
percentages were derived from counts taken during observations. The other 
indices of student behavior were derived from classroom observation ratings. 
This table shows that for most of the key student variables, the behavior of 
the students in the experimental group teachers' classrooms was significantly 
better than the students in the control group classrooms. 

12. Please provide actual means and standard deviations for Tables 4, 5, 6 6. 

These scores are provided on the attached sheets for Tables 4, 5, & 6. 
Please note that only the means, not the standard deviations, are available for 
Studies 3 & 4. These data were reported using only the means. Both means and 
standard deviations are given for Studies 5 ft 6, however. 

Please note that Table 5 of the submittal had a few typographical errors 
in representing the significance levels. The means, standard deviations, and 
significance levels shown in supplementary Tables 5a, 5b, 6a, and 6b are 
accurate. Ve apologize for any inconvenience. 

13. (See pp. 13 & 15) Bow are the negatives handled on Tables 5 & 6? 

The negatives simply mean that the experimental groups had significantly 
lower scores than the control groups on these variables. Since these variables 
tend to indicate undesirable conditions in the classrooms (high amounts of 
inappropriate behavior, off-task behavior, students wandering around, downtime, 
etc) , lower scores in the experimental groups are in the hypothesized 
direction. 



er|c 23 



14. (See p. 10, Xaatruaaata tad procedures) ftefereace la aada to 3 taita uatd 
la the atudy; however, results ara profidad for oaly 1 taat. Plaaaa provide 
tha raaulta for tba othar two taata. 

Descriptions of the three tests are as follows: 

(1) analyses of student raw gains and regression adjusted performance 
(ANCOVA) pooled across classrooms; 

(2) analyses of between-class variance on raw gain and regression-adjusted 
gain (ANCOVA); and 

(3) paired comparisons, class-by-class, of the two pairs of classes with 
SATBS pre-scores and SRA post-scores (ANCOVA and, for reading and language 
arts, multivariate ANCOVA). 

Means and f -ratios for each of the three tests are shown in Table 4, attached. 



Table 4. Analyses Using Three Teats fcr Student Achievcnmt Differences (Study 5) 



Bcperiaental Control 

M SD N M SD N f-ratio fi 



TBI 1; CRT Scares Across Classes 
Raw (Seine: 

Language arts 10.72* 6.7S (29) -0.79 10.U (89} 32.82 < .0001 

Math 5.53* 19.1? (34) -2.97 14.48 (34) 4.26 < .05 

ANOOVA - Adjusted Post-test Mans: 

Language arts 54.41* 43.00 32.74 < .001 

Math 50.40* 44.95 2.24 .14 
TBT 2: Off Scores Bet wee n Class Analyses 



Haw Cains. 

Language arts 10.73* 0.02 (29) -0.80 2.84 (89) 479.71 < .0001 

Math 5.53* 2.83 (34) -2.97 3.79 (34) 110.04 < .0001 

AMOOVA - Adjusted Post-test Means: 

Language arts 54.25* 43.05 504.87 < .001 

Math 59.61* 35.75 894.51 < .001 

TBT 3: Paired Class-by-Class Oosparisons 

AMOOVA - Adjusted Post-test Means: 

'Language arts 33.86* (19) 29.91 (16) 1.64 .34 
(2 classes) 

Reading 31.31 35.60* .93 .21 

(sane 2 classes as LA above) 

Math (2 classes) 26.06 (14) 27.63* (12) .20 .66 



* Indicates the higher nean score 

1 For these two classes both Language arts and readings scores were analyzed. 
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Table 5a. Means far classroom ratings of teacher behavior and teaching practice in two elemental field studies 
(Studies 3 & 4). 



IX. Developing Rules and 



Study 3 study 4 

cms jms 

* 1-6 Gr 6-8 

N-41 N>38 

Ex Con p Ex Con p 

M M < M H < 



Teacher Behavior Variables 
I. Readying the Classroom 

Classrooa is ready for 

school 4.28 3.71 .05 4.28 3.90 (.07) 



a. Efficient administrative 

routines 3.92 3.55 ^05 4.14 3.75 ^01 

b. Appropriate general 

procedures 3.92 3.33 .01 3.88 3.43 .05 

c. Efficient snail group 3.61 3.56 NS - 
procedures 

m. Monitoring 

a. Of student behavior 3.91 3.29 .01 3.87 3.10 .001 

b. Of transitions between 3.72 2.89 .05 3.64 3.08 .05 
activities 

IV. Managing Student Behavior 

a. Rewards good perfornance 3.35 2.94 (.08) 2.50 1.94 .05 

b. Consistent management of 3.90 3.11 .01 3.70 3.14 .05 
behavior 

c. Signals appropriate behavior 3.47 2.74 .01 - 

d. Allows class to get 

out of hand - 1.68 2.51 .05 



Table 5a. cont'd 



Study 3 

cms 

Ge 1-6 
IM1 

fie Con p, 

M M < 



Study 4 
JMIS 
<Sr 6-8 
N-38 
fit Con 
M M 



P 

< 



e. Stops disruptive behavior 
quickly 

£. Stops inappropriate be- 
havior quickly 



4.39 
3.73 



a. Presents, reviews, disauww 3.51 
rules/procedures 

b. Rehearsal, practice- included 3.15 
in presentation 

c. Teacher stays in charge of 4.32 
whole class 

vi. Orgwrirlng Instruction 

a. Student attention spans 3.57 
considered 

b. Appropriate lesson pacing 3.69 

c. Lessons related cO students' 
backgrounds and interests 

d. Low anoint of down tiae: 
students waiting 

vn. Student Accountability 

a. Checks students' 3.80 
understanding 

b. Consistency in enforcing 3.92 
acadenic work standards 

c. Suitable routines for 3.83 
collecting 4 checking work 



3.27 
2.84 



2.64 
2.20 
3.53 



3.38 



3.47 



3.46 
3.23 
3.40 



.01 



g. Ignores inappropriate behavior 2.30 2.97 .01 

v. Teaching Rules & Procedures 
(first week of school) 



J6 
^01 
.05 



4.23 
3.86 

2.25 



3.09 



4.59 



3.62 



3.64 



3.50 .05 
3.18 i01 

2.89 .01 



2.61 (.06) 



1.96 1.43 (.07) 



4.38 MS 



3.28 (.06) 



3.37 NS 



3.72 3.19 



3.68 3.12 



3.85 3.51 



.01 



.01 



.05 



Table 54. cont'd 



Study 3 study 4 

QHS MS 

* 1-6 Gr 6-8 

N-41 N»38 





Be 
H 


Con 
N 


P 

< 


EX 
M 


Con 
M 


P 

< 


d. Teacher holds students 
responsible for academic work 


4.00 


3.33 




4.13 


3.55 




e. Monitors progress in 
completing assignment* 


4.09 


3.32 


±05 


3.83 


3.33 


iQ5 


f . Class has task-oriented focus 


3.98 


3.42 


±05 


3.79 


3.41 


^05 


g. Teacher plans appropriate 
amount of class work 


4.07 


3.21 




4.47 


3.72 


.001 


vm. LjstructicDil Clarity 














a. Describes objectives clearly 


3.39 


2.91 


±05 


3.35 


3.05 


NS 


b. Clear directions for 
assignments 


3.93 


3.52 


(.06) 


3.91 


3.48 


(.09) 


c. Clear explanations & 
presentations 


4.04 


3.55 


±05 


3.77 


3.49 


MS 


d. Checks students' understand- 
ing of instructions 


3.63 


2.94 


±PI 


3.72 


3.19 


±91 



Variables are five-point scales: l*least characteristic or least frequent; 5-oost characteristic or oost frequent. 
HS: not significant 

Significance levels far Studies 3 & 4 are based on two-tailed tests. 
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Table 5b. Means and standard deviations for classroom ratings of teacher behavior and teaching practice in tuo 
experimental field studies (Studies 5 and 6) . 



Study 5 study 6 

QflEP CMTST 
Gr 1-6 fir 7-9 

N«29 N.16 

Be Con p effect Ex Con e effect 

M SO M SD < size M SD M SO < size 



Teacher Behavior Variables 
I. toadying the diserooB 

Classroom is ready for 
school 

n. Developing Rules and 



4.54 .88 4.07 .88 (.09) .53 4.69 .53 3.94 1.12 (.06) .67 



a. Efficient administrative 

routines 4.55 .52 3.57 .92 .001 



b. Appropriate general 
procedures 

c. Efficient small group 
procedures 

in. Monitoring 

a. Of student behavior 



4.41 .56 3.68 .72 .001 
4.04 1.07 2.63 1.06 .01 

4.35 .64 3.64 1.13 .05 



b. Consistent management of 
behavior 

c. Signals appropriate 
behavior 

d. Allows class to get 
out of hand 



4.58 .88 4.42 .89 MS 



4.57 .82 4.14 1.00 . 05 



b. Of transitions between 4.58 .86 4.00 1.07 .05 
activities 

IV. Nmgiag Student Behavior 

a. Rewards good performance 3.66 .97 2.77 . 97 . 05 



4.05 1.23 3.33 .91 .05 
4.81 .53 3.81 1.13 .05 



3.93 1.16 3.08 1.20 . 05 

4.40 .58 3.51 1.05 .05 4.06 1.20 2.97 1.17 JQ5 

3.58 1.12 2.74 1.21 .01 3.27 .86 2.08 1.01 .01 

1.54 1.13 1.93 1.28 NS .30 1.31 .70 2.3/ 1.38 . 05 
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Table 5b. (cont'd) 



Study 5 study 6 

OflW CMTST 
Cr 1-6 Gr 7-9 

N°29 11=16 

Ex Con g effect Ex Ceo e effect 

M SD M SD < size M SD M SD < size 



e. Stops disruptive behavior 

quickly 4.85 .38 4.60 .74 NS .33 4.63 .88 3.31 1.41 ^05 

f . Stops inappropriate be- 
havior quickly 3.76 .87 2.85 1.00 ip5 4.63 .87 3.31 1.41 ip5 

g. Ignores inapprop. beh. 2.19 1.03 2.74 .91 NS 2.51 1.53 4.10 1.10 .05 

v. Teaching Rules 6 Frw*«ures 
dint week of school) 

a. Presents, reviews, discusses 

rules/procedures 4.11 1.05 2.38 1.51 .05 4.00 1.85 3.71 1.89 NS 

b. Rehearsal, practice 4.56 .88 3.00 1.58 ^05 4.38 1.40 2.71 1.80 J£ 
included in presentation 

c. Teacher & fc ays in charge 

of whole class 5.00 .00 A.-2 .47 .05 4.50 1.41 4.00 1.73 NS 

vi. organizing Instruction 

a. Student attention spans 

considered 4.15 .59 3.43 . 73 M 3.87 .83 3.28 .79 .05 

b. Appropriate lesson pacing 4.31 .62 3.32 1.05 ^01 4.15 .87 3.41 .66 M 

c. Lessons related to students' 

backgrounds & interests 3.51 .72 2.96 .88 NS .62 3.82 .86 3.11 1.01 ^05 

d. Low amount of down tine: 

students waiting 4.38 .96 3.73 .96 .05 3.75 .85 2.88 1.51 (.10) .59 

vn. student Accountability 

a. Checks students' 

understanding 4.36 . 65 3.58 1.07 .05 4.46 .70 3.92 1.02 ^01 

b. Consistency in enforcing 

academe work standards 4.09 .89 3.62 1.01 (.10) .46 4.27 .80 3.41 .89 ^05 

c. Suitable routines for col- 
lecting & checking work 4.20 .57 3.63 .80 ^05 4.53 .74 4.20 .83 .05 
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Table Sb. (cont'd) 



Study 5 Study 6 

OfBT OUST 

1-6 Or 7-9 

JW9 N>16 

Be Con e Be Con p effec 

M SD M SD < size M SD M SD < sue 

d. Teacher holds students respon- 
sible for acad emic work 4.62 . 65 3.86 1.07 .05 4.56 .62 3.75 .80 ±05 

e. Monitors progress in 

CQBpleting assignments - -- -- .... 

f . Class has task-oriented 

fecus 4.42 . 56 3.83 .95 ±05 . 4.53 .73 3.85 . 75 ±01 

g. Teacher plans appropriate 

aoount of class work 5.00 .00 4.53 .64 ±01 4.94 1.09 4.13 1.03 ±01 

vm. Instructional Clarity 

a. Descr i bes objectives 

dearly 3.99 1.25 2.94 1.22 ±05 4.95 .10 4.27 .83 ±05 

b. Clear directions for 

assignments 4.43 .63 3.90 . 93 ±05 4.66 .63 4.15 .85 ±05 

c. dear explanations & 

presentations 4.51 .47 3.76 .79 ±05 4.45 .81 3.85 . 91 ±01 

d. Checks students' under- 
standing of instructions 4.41 .63 3.56 1.02 ±05 4.47 .72 3.90 .99 ±01 



Variables are five-point scales: l*least characteristic or least frequent; 5rost characteristic or most frequent, 
es: effect size 
KS: not significant 

Significance levels for Studies 5 & 6 are based on one-tailed tests. 
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Table 64. Mean fran observational data on student behavior collected in two experimental field 
studies (Studies 3 4 4). 







Study 3 




Study 4 












CXES 




JMXS 












Or 1-6 




Gr 6-8 












N-41 




N-38 










SI 


a — 

con 


_ , mm . _^ 

2 effect 


fie 


Con 




effect 




M 


M 


< size 


X 


M 


< 


size 


Student Variables 
















* BfcmlMi i ■ nil 1 rait 

a. students caii-out 
















frequently 


2.35 


2.94 




2.01 


2.91 






b. Students are frequently 
















out 01 seats wandering 








1.57 


2.28 


.05 




c. Low amount of inappro- 
















priate behavior in class 


2.61 


3.09 




2.13 


2.63 


NS 




d. % of students on-task 


92.49 


88.93 




91.00 


85.00 






e. % of students off-task 


3.04 


5.01 




4.00 


6.00 


^05 




f . Student success in class 
















lessons 


3.80 


3.66 


NS 


4.05 


3.77 


(.10) 





Variables a, b, c, & f are five-point scales: l=least characteristic or least frequent; 5anost 

characteristic or most frequent. 

Significance levels are based on two-tailed tests. 



Table 6b. Means and standard deviations fro. observational data on student behavior collected in two 
experimental field studies (Studies 5 ft 6). 

Study 5 Study 6 

OfECr CMTST 

Cr 1-6 Gr 7-9 

K-29 tH6 

Be Con g Be Con 2 

H SD M SD < M SD M SD < 

S*«frn» Variable 

a. Students call-out 

frequently 2.00 .72 2.45 . 90 MS 1.69 .88 2.60 1.00 & 

b. Students are frequently 

out of seats rendering 1.74 .91 2.23 1.21 (.10) 1,38 . 69 2.25 1.31 ^05 

c. Low amount of inappro- 
priate behavior in class 1.69 .58 2.49 1.03 ^05 1.95 .95 2.76 1.09 ^05 

d. % of students co-task 93.20 9.13 82.70 22.66 (.10) 87.95 10.95 75.53 12.44 & 

e. % of students off-task 3.14 2.83 12.23 14.56 ^05 7.09 7.86 14.79 10.32 .05 

f . Student success in class 

lessens 3.82 .45 3.46 .64 (.10) 4.36 .52 3.85 .61 .01 



Variables a, b, c, ft f *ce five-point scales: l>leaat characteristic or least frequent; 5*oost 
characteristic or most frequent. 

Significance levels for Studies 5 ft 6 are based on one-tailed tests. 
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